feat: add perf support in walltime by not-matthias · Pull Request #73 · CodSpeedHQ/codspeed

not-matthias · 2025-03-25T11:15:48Z

TODOs:

~~Don't use max freq (or parse it and send it with the traces)~~ Actually doesn't matter. Higher freq, better callgraph.
upload data + debug symbols to server
~~use correct codspeed version (depends on Cod 674 collect profiles while benchmarks are running codspeed-rust#92)~~

Debug logs of successful run:

►►► Running the benchmarks
[DEBUG::codspeed::run::runner::helpers::setup] Running command: sudo sysctl -w kernel.kptr_restrict=0
[DEBUG::codspeed::run::runner::helpers::setup] Running command: sudo sysctl -w kernel.perf_event_paranoid=1
[DEBUG::codspeed::run::runner::wall_time::executor] cmd: ARCH="x86_64" CODSPEED_ENV="runner" CODSPEED_PROFILE_FOLDER="/tmp/profile.riowzUmLRF.out" CODSPEED_RUNNER_MODE="walltime" PYTHONHASHSEED="0" "sh" "-c" "perf record --data --freq=1000 --switch-output --control=fifo:/tmp/codspeed_perf.ctl.fifo,/tmp/codspeed_perf.ack.fifo --delay=-1 -g --output=/tmp/.tmpeI44Iq/perf.datao1Dpk6 -- cargo-codspeed run"
[DEBUG::codspeed::run::runner::wall_time::executor] Perf PID: 527912
Events disabled
[cargo-codspeed] Measurement mode: Walltime

Collected 1 benchmark suite(s) to run
Running graph-analysis my_bench
Timer precision: 13 ns
my_bench        fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ fibonacci                  │               │               │               │         │
Events enabled
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump /tmp/.tmpeI44Iq/perf.datao1Dpk6.2025032512120419 ]
Events disabled
│  ╰─ 32        4.563 ms      │ 7.109 ms      │ 4.881 ms      │ 5.07 ms       │ 100     │ 100
╰─ other_bench                │               │               │               │         │
Events enabled
[ perf record: dump data: Woken up 2 times ]
[ perf record: Dump /tmp/.tmpeI44Iq/perf.datao1Dpk6.2025032512120472 ]
Events disabled
   ╰─ 50        50.06 ms      │ 50.31 ms      │ 50.1 ms       │ 50.12 ms      │ 100     │ 100

Done running   ⠉ Running the benchmarks...                                                                                                                                   my_bench
Finished running 1 benchmark suite(s)
[ perf record: Woken up 2 times to write data ]
[ perf record: Dump /tmp/.tmpeI44Iq/perf.datao1Dpk6.2025032512120974 ]
[ perf record: Captured and wrote 0.034 MB /tmp/.tmpeI44Iq/perf.datao1Dpk6.<timestamp> ]
  ⠉ Running the benchmarks...                                                                                                                                                [src/run/runner/wall_time/executor.rs:117:9] &perf_files = [
    "/tmp/.tmpeI44Iq/perf.datao1Dpk6.2025032512120419",
    "/tmp/.tmpeI44Iq/perf.datao1Dpk6.2025032512120472",
    "/tmp/.tmpeI44Iq/perf.datao1Dpk6.2025032512120974",
]

art049

A first quick review, I'll try to review more thouroughly

src/run/runner/wall_time/perf.rs

Cargo.toml

GuillaumeLagrange

Good job overall ! Some minor details

src/run/runner/helpers/setup.rs

src/run/runner/wall_time/executor.rs

src/run/runner/helpers/run_command_with_log_pipe.rs

src/run/runner/wall_time/executor.rs

src/run/runner/wall_time/perf.rs

src/run/runner/wall_time/executor.rs

src/run/runner/wall_time/perf.rs

art049

looking good overall :)

src/run/runner/wall_time/perf.rs

src/run/runner/wall_time/executor.rs

Copilot

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/run/runner/wall_time/perf/fifo.rs:168

Consider including the actual pid value in the error message to provide clearer context when the process lookup fails.

let bench_proc = procfs::process::Process::new(pid as _).expect("Failed to find benchmark process");

src/run/runner/wall_time/perf/helpers.rs

GuillaumeLagrange

Great piece of work! Some overall small changes, next should be the last round of review!

src/run/runner/mod.rs

src/run/runner/wall_time/executor.rs

src/run/runner/wall_time/perf/helpers.rs

src/run/runner/wall_time/perf/mod.rs

src/run/runner/wall_time/perf/perf_map.rs

src/run/runner/wall_time/perf/shared.rs

src/run/runner/wall_time/perf/unwind_data.rs

src/run/runner/wall_time/perf/shared.rs

src/main.rs

src/run/runner/wall_time/executor.rs

GuillaumeLagrange

Very minor comments appart from the multi-threading stuff we discussed IRL

src/run/runner/wall_time/perf/helpers.rs

src/run/runner/wall_time/perf/mod.rs

not-matthias · 2025-05-07T08:10:35Z

@GuillaumeLagrange Only minor changes in this PR: Added conditional dependency of procfs, disabled USE_PERF by default, install perf on setup if not found.

GuillaumeLagrange

lgtm, good work!

See COD-843 for more information.

not-matthias requested a review from GuillaumeLagrange March 25, 2025 11:15

art049 reviewed Mar 25, 2025

View reviewed changes

src/run/runner/wall_time/perf.rs Outdated Show resolved Hide resolved

not-matthias commented Mar 26, 2025

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

GuillaumeLagrange requested changes Mar 26, 2025

View reviewed changes

GuillaumeLagrange reviewed Mar 26, 2025

View reviewed changes

src/run/runner/wall_time/perf.rs Outdated Show resolved Hide resolved

GuillaumeLagrange reviewed Mar 26, 2025

View reviewed changes

src/run/runner/wall_time/executor.rs Outdated Show resolved Hide resolved

src/run/runner/wall_time/perf.rs Outdated Show resolved Hide resolved

art049 approved these changes Mar 28, 2025

View reviewed changes

src/run/runner/wall_time/perf.rs Outdated Show resolved Hide resolved

src/run/runner/wall_time/executor.rs Outdated Show resolved Hide resolved

not-matthias requested review from GuillaumeLagrange and Copilot April 7, 2025 13:15

Copilot AI reviewed Apr 7, 2025

View reviewed changes

src/run/runner/wall_time/perf/helpers.rs Show resolved Hide resolved

not-matthias requested a review from art049 April 8, 2025 15:54

GuillaumeLagrange requested changes Apr 11, 2025

View reviewed changes

GuillaumeLagrange closed this Apr 11, 2025

GuillaumeLagrange deleted the cod-674-collect-profiles-while-benchmarks-are-running branch April 11, 2025 06:19

GuillaumeLagrange restored the cod-674-collect-profiles-while-benchmarks-are-running branch April 11, 2025 06:19

GuillaumeLagrange reopened this Apr 11, 2025

not-matthias force-pushed the cod-674-collect-profiles-while-benchmarks-are-running branch 3 times, most recently from 95c91a1 to 6d4a041 Compare April 22, 2025 09:06

not-matthias requested a review from GuillaumeLagrange April 23, 2025 12:28

art049 reviewed Apr 25, 2025

View reviewed changes

src/run/runner/wall_time/perf/unwind_data.rs Outdated Show resolved Hide resolved

art049 reviewed Apr 25, 2025

View reviewed changes

src/run/runner/wall_time/perf/shared.rs Outdated Show resolved Hide resolved

art049 reviewed Apr 25, 2025

View reviewed changes

src/main.rs Outdated Show resolved Hide resolved

not-matthias force-pushed the cod-674-collect-profiles-while-benchmarks-are-running branch from 6d4a041 to 8fca095 Compare April 25, 2025 09:04

GuillaumeLagrange reviewed Apr 25, 2025

View reviewed changes

src/run/runner/wall_time/executor.rs Show resolved Hide resolved

GuillaumeLagrange approved these changes Apr 25, 2025

View reviewed changes

src/run/runner/wall_time/perf/helpers.rs Outdated Show resolved Hide resolved

src/run/runner/wall_time/perf/helpers.rs Outdated Show resolved Hide resolved

src/run/runner/wall_time/perf/mod.rs Outdated Show resolved Hide resolved

not-matthias force-pushed the cod-674-collect-profiles-while-benchmarks-are-running branch from 8fca095 to 1b67cd8 Compare April 25, 2025 13:00

chore: dont use regex in perf map harvest

9e9db5b

adriencaccia force-pushed the cod-674-collect-profiles-while-benchmarks-are-running branch 2 times, most recently from bec9d0a to 26b46ad Compare April 29, 2025 08:03

not-matthias force-pushed the cod-674-collect-profiles-while-benchmarks-are-running branch 3 times, most recently from 94bebd2 to e43a2ff Compare May 1, 2025 12:51

feat: add fifo ipc

0790357

not-matthias force-pushed the cod-674-collect-profiles-while-benchmarks-are-running branch 3 times, most recently from 80d0929 to 73a079c Compare May 7, 2025 07:53

not-matthias requested a review from GuillaumeLagrange May 7, 2025 08:09

GuillaumeLagrange approved these changes May 7, 2025

View reviewed changes

not-matthias and others added 5 commits May 7, 2025 10:30

feat(runner): add perf integration for rust

3871787

feat(runner): add perf integration for python

476a551

chore: Release codspeed-runner version 3.6.0-beta.2

308bb0f

feat: install perf on setup

8e4a390

fix: use bash to ensure correct behavior across systems

4d6c062

See COD-843 for more information.

not-matthias force-pushed the cod-674-collect-profiles-while-benchmarks-are-running branch from 73a079c to 4d6c062 Compare May 7, 2025 08:31

not-matthias merged commit 4d6c062 into main May 7, 2025
9 checks passed

not-matthias deleted the cod-674-collect-profiles-while-benchmarks-are-running branch May 7, 2025 08:47

Conversation

not-matthias commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

art049 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

GuillaumeLagrange left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

art049 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

GuillaumeLagrange left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GuillaumeLagrange left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

not-matthias commented May 7, 2025

Uh oh!

GuillaumeLagrange left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

not-matthias commented Mar 25, 2025 •

edited

Loading

GuillaumeLagrange left a comment •

edited

Loading